Merged
Conversation
…or DB storage When the messages or response JSON fields in spend logs are truncated before being written to the database, the truncation marker now includes a note explaining: - This is a DB storage safeguard - Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.) - The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit Also emits a verbose_proxy_logger.info message when truncation occurs in the request body or response spend log paths. Adds 3 new tests: - test_truncation_includes_db_safeguard_note - test_response_truncation_logs_info_message - test_request_body_truncation_logs_info_message Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
|
Cursor Agent can help with this pull request. Just |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Contributor
Greptile SummaryThis PR adds a human-readable explanation ( Key observations:
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/constants.py | Adds LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE constant (~201 chars) placed alongside existing truncation constants. Straightforward and self-contained. |
| litellm/proxy/spend_tracking/spend_tracking_utils.py | Appends safeguard note to the in-value truncation marker, inflating stored DB field sizes beyond the intended MAX_STRING_LENGTH_PROMPT_IN_DB limit. Contains redundant local import of constants already imported at module level. The truncation-detection logic and info-log feature work as intended. |
| tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py | New tests cover safeguard note presence and info-log emission on truncation; all mock-based and locally runnable. One test assertion contains a redundant branch that can be simplified. |
Last reviewed commit: b211407
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+635
to
+638
| from litellm.constants import ( | ||
| LITELLM_TRUNCATED_PAYLOAD_FIELD, | ||
| LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE, | ||
| ) |
Contributor
There was a problem hiding this comment.
These constants are already imported at the module level (lines 14–17). This local import is now redundant and can be removed.
Suggested change
| from litellm.constants import ( | |
| LITELLM_TRUNCATED_PAYLOAD_FIELD, | |
| LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE, | |
| ) | |
| if visited is None: |
Comment on lines
682
to
688
| truncated_value = ( | ||
| f"{value[:start_chars]}" | ||
| f"... ({LITELLM_TRUNCATED_PAYLOAD_FIELD} skipped {skipped_chars} chars) ..." | ||
| f"... ({LITELLM_TRUNCATED_PAYLOAD_FIELD} skipped {skipped_chars} chars. " | ||
| f"{LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE}) ..." | ||
| f"{value[-end_chars:]}" | ||
| ) | ||
| return truncated_value |
Contributor
There was a problem hiding this comment.
The truncation marker now includes LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE (~201 characters), which means every truncated field stored in the database will exceed the intended MAX_STRING_LENGTH_PROMPT_IN_DB limit by approximately 270+ characters. With a default limit of 2048, this represents a ~13% size increase per truncated field—the opposite of the intended safeguard.
The safeguard note is already surfaced in logs via the new verbose_proxy_logger.info() calls (lines 803–806 and 886–889), making the embedded text redundant. Consider either:
- Storing the note in a separate metadata field
- Keeping the in-value marker minimal and relying on the log-level info messages
| assert LITELLM_TRUNCATED_PAYLOAD_FIELD in truncated | ||
| assert LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE in truncated | ||
| assert "DB storage safeguard" in truncated | ||
| assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower() or "logging callbacks" in truncated |
Contributor
There was a problem hiding this comment.
The third or condition is redundant. Since LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE already contains the lowercase phrase "logging callbacks", the expression "logging callbacks" in truncated will always be True if "logging callbacks" in truncated.lower() is True.
Suggested change
| assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower() or "logging callbacks" in truncated | |
| assert "logging callbacks" in truncated.lower() or "logging integrations" in truncated.lower() |
krrishdholakia
added a commit
that referenced
this pull request
Mar 6, 2026
…verflow (#22950) * feat(spend-logs): add truncation note when error logs are truncated for DB storage (#22936) When the messages or response JSON fields in spend logs are truncated before being written to the database, the truncation marker now includes a note explaining: - This is a DB storage safeguard - Full, untruncated data is still sent to logging callbacks (OTEL, Datadog, etc.) - The MAX_STRING_LENGTH_PROMPT_IN_DB env var can be used to increase the limit Also emits a verbose_proxy_logger.info message when truncation occurs in the request body or response spend log paths. Adds 3 new tests: - test_truncation_includes_db_safeguard_note - test_response_truncation_logs_info_message - test_request_body_truncation_logs_info_message Co-authored-by: Cursor Agent <cursoragent@cursor.com> * Fix admin viewer unable to see all organizations The /organization/list endpoint only checked for PROXY_ADMIN role, causing PROXY_ADMIN_VIEW_ONLY users to fall into the else branch which restricts results to orgs the user is a member of. Use the existing _user_has_admin_view() helper to include both roles. * feat(ui): add Chat UI — ChatGPT-like interface with MCP tools and streaming (#22937) * feat(ui): add chat message and conversation types * feat(ui): add useChatHistory hook for localStorage-backed conversations * feat(ui): add ConversationList sidebar component * feat(ui): add MCPConnectPicker for attaching MCP servers to chat * feat(ui): add ModelSelector dropdown for chat * feat(ui): add ChatInputBar with MCP tool attachment support * feat(ui): add MCPAppsPanel with list/detail view for MCP servers * feat(ui): add ChatMessages component; remove auto-scrollIntoView that caused scroll-lock bypass * feat(ui): add ChatPage — ChatGPT-like UI with scroll lock, MCP tools, streaming * feat(ui): add /chat route wired to ChatPage * feat(ui): remove chat from leftnav — chat accessible via navbar button * feat(ui): add Chat button to top navbar * feat(ui): add dismissible Chat UI announcement banner to Playground page * feat(proxy): add Chat UI link to Swagger description * feat(ui): add react-markdown and syntax-highlighter deps for chat UI * fix(ui): replace missing BorderOutlined import with inline stop icon div * fix(ui): apply remark-gfm plugin to ReactMarkdown for GFM support * fix(ui): remove unused isEvenRow variable in MCPAppsPanel * fix(ui): add ellipsis when truncating conversation title * fix(ui): wire search button to chats view; remove non-functional keyboard hint * fix(ui): use serverRootPath in navbar chat link for sub-path deployments * fix(ui): remove unused ChatInputBar and ModelSelector files * fix(ui): correct grid bottom-border condition for odd server count * fix(chat): move localStorage writes out of setConversations updater (React purity) * fix(chat): fix stale closure in handleEditAndResend - compute history before async state update * fix(chat): fix 4 issues in ChatMessages - array redaction, clipboard error, inline detection, remove unused ref * docs: add PayGo/priority cost tracking for Gemini Vertex AI - Add PayGo / Priority Cost Tracking section to Vertex AI provider docs - Document trafficType to service_tier mapping (ON_DEMAND_PRIORITY, FLEX, etc.) - Add service tier cost keys to custom pricing docs - Add provider-specific cost tracking note to spend tracking overview Made-with: Cursor * fix: normalize response images missing index + guard audio duration overflow 1. convert_dict_to_response.py (#22640): Providers like OpenRouter/Gemini return images without the required `index` field, causing pydantic ValidationError when constructing Message. Added _normalize_images() to backfill index from enumeration position. 2. audio_utils/utils.py (#22622): libsndfile can report 2^63-1 frames for malformed audio files, causing astronomically large duration values used for cost calculation. Added guards for sentinel frame counts and implausible durations (>24h). Co-Authored-By: claude-flow <ruv@ruv.net> * fix: add type annotations to _normalize_images + guard samplerate==0 Address review feedback: - Add type hints to _normalize_images() for consistency with codebase - Guard against samplerate <= 0 to prevent ZeroDivisionError on malformed audio files Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Ryan Crabbe <rcrabbe@berkeley.edu> Co-authored-by: ryan-crabbe <128659760+ryan-crabbe@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: claude-flow <ruv@ruv.net>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
✅ Test
Changes
Adds a note to truncated log messages explaining that truncation is a DB safeguard and that full, untruncated data is sent to logging callbacks (e.g., OTEL, Datadog). This clarifies why certain log fields might appear truncated in the database while being complete in other logging systems.
LITELLM_TRUNCATION_DB_SAFEGUARD_NOTEconstant._sanitize_request_body_for_spend_logs_payloadto include this safeguard note in the truncation marker.